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1. INTRODUCTION 

Spectrum sensing is usually done by measuring the power spectral density of the channel of interest 
in cognitive radio [1] based system to perform the dynamic spectrum access [2] operation. This method 
doesn’t give estimated value or doesn’t help in prediction in a noisy channel. Estimating the status of channel 
as being free or busy is important aspect of cognitive radio system. A hidden Markov process (HMP) [3-6] is 
useful in this prediction. A hidden Markov process (HMP) is a discrete-time finite state homogeneous 
Markov chain (MC) [7] observed through a discrete-time memory less invariant channel. HMPs are more 
commonly referred to as hidden Markov models [3]. The parameter of HMP is estimated offline from real 
measurements using Baum-Welch (B-W) algorithm [8, 9] and maximum likelihood (ML) algorithm [10]. 
Giventhisparameters the state of the primary user at a given time and frequency band is determined. Hidden 
Markov model (HMM) [11-24] is a statistical Markov model in which the system being modeled is assumed 
to be a Markov process with unobservable (i.e. hidden) states. Here we avoided mathematical expressions 
and formulas and provided only experimental data and results. 

The rest of the paper is organized as follows. In section 2, system model is explained. In section 3, 
we find the most likely states using Viterbi algorithm [25] and compare it with original states of the channel. 
In section 4, we compare the Baum-Welch and maximum likelihood algorithm and concluded in section 5. 
We used @MATLAB for this experimentation. 
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2. SYSTEM MODEL 

Let us consider HMP@ = (z,A,B), parameterized by the initial state matrix m, the state transition 
matrix A, and the emission matrix B.; Here we assume a discrete-time model, where the time variable n takes 
values in {1,2,...}. The input to the system at time n consists of an estimate of the power spectral density. 
Each channel is modeled as a two-state HMP. The output of the system consists of an estimated transition 
and emission probability matrices of each channel from which we can predict the current states as well as 
the future states of the channel using predictor. 

Channel parameter estimation using HMP, associate with each of the channels is based on offline 
training data. HMP may be view as a discrete time bivariate random process {(Sn, Yn), n=1,2,...}, where {Sn} 
is the hidden process i.e. states of the channel and {Yn} is the observable process i.e. sequences. The hidden 
process {Sn} is a discrete time finite state Markov chain. The random variables {Yn; are conditionally 
independent given {Sn}. Furthermore, the distribution of Yn is time invariant and depends on {Sn} only 
through Sn. The hidden process {Sn} takes values in a finite set S = { 0,1, ... M-1}. In this system we have 
focused on the case M=2, such that 1 represents the channel is free whereas state 2 represents a primary user 
existence on the channel. Figure | shows the state-transition diagram of the hidden process. 














Figure 1. The state-transition diagram of a hidden process. 


3. FINDING MOST LIKELY STATES 

Using transition and emission matrices of the channel, we generated the sequences and states of 
the channel using @MATLAB. Using Viterbi algorithm, we generate the most likely states and compare it 
with channel states. Figure 2 shows the block diagram we used to check Viterbi algorithm. We create 
the Hidden Markov Model (HMM) using transition probability matrix and emission probability matrix. 
We create two HMM, HMM1 and HMM2 having same emission probability matrix considering the same 
communication system.HMM1 is monotonous while HMM2 is dynamic where states changes more fast. 
Here Trans! and Trans2 are transition matrices for HMM1 and HMM2 respectively. Emis is the common 
emission probability matrix. We used, Trans! = [0.95, 0.05; 0.10, 0.90], Trans2 = [0.05, 0.95; 0.90, 0.10] and 
Emis = [0.25, 0.20, 0.10, 0.20, 0.25; 0.05, 0.2, 0.5, 0.2, 0.05]. Here Viterbi algorithm is used to find 
the hidden states from the given sequence. We created ten sequences and its states from same HMM and 
using Viterbi algorithm found out estimated states. On an average estimated state sequences are 90.66 % 
correct for HMM1 and 84.8 % correct for HMM2. Table 1 shows the sample data for estimating states using 
Viterbi algorithm. 

Figure 3 and Figure 4 shows the eigenvalues of transition probability matrix of HMM1 and HMM2 
respectively. An eigenvalue plot indicates whether the Markov chain is periodic, and the plot reveals 
the period of the chain. All eigenvalues at roots of unity indicate the periodicity. The spectral gap is the area 
between the circumference of the unit circle and the circumference of the circle with a radius of the second 
largest eigenvalue magnitude (SLEM). The size of the spectral gap determines the mixing rate of the Markov 
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chain. The mixing time of a Markov chain is the time until the Markov chain is "close" to its steady 
state distribution. In general, the spectrum determines structural properties of the chain. Here we simply 
exchange the occurrence probabilities of both states so spectral gap for both the HMMs are same but 
eigenvalues are different. A stationary distribution of a MC is a probability distribution that remains 
unchanged in a MC as it progresses. It give us the important confirmations like transient state and recurrent 
stat. It represents the limiting time-independent distribution of the state for Markov process as the number of 
steps on transition increases. 






Sequences 


Viterbi Algorithm 


Most likely States 


Figure 2. Block diagram to check Viterbi Algorithm 








Table 1. Sample data for estimating States using Viterbi algorithm. 
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Figure 3. Eigenvalues of HMM 1on complex planes 
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Figure 4. Eigenvalues of HMM2 on complex planes 


4. COMPARISON OF BAUM-WELCH AND MAXIMUM LIKELIHOOD ALGORITHM 

Here we used the following block diagram (Figure 5) to conduct the comparison. For Baum-Welch 
algorithm, transition probability matrix and emission probability matrix along with sequences generated by 
them have been used to find out the estimated matrices. While for Maximum Likelihood algoritham, 
sequences generated and states generated by these matrices have been used. 














States 








Baum-Welch Maximum Likelihood 
Algorithm Estimation 


EST-TRAN EST-TRAN 
EST-EMIS EST-EMIS 


Figure 5. Block diagram to find comparison of two algorithms 








4.1. Estimation using Baum-Welch algorithm 

Here we estimate the transition and emission probability matrixes using Baum-Welch algorithm. 
Using MATLAB, we estimated the matrixes using the sequences and transition probability matrix and 
emission probability matrix of the system. By running the experiments six times we found that averages of 
the matrixes gives better results than single iteration. Table 2 shows the result we obtain for HMM1. Here 
estTR is the sample estimated transition matrix and estEmis is the sample estimated emission matrix, while 
AvEstimatedTran is the average estimated transition matrix and AvEstimatedEmis is the average estimated 
emission matrix. Table 3 shows the result we obtain for HMM2. 
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Table 2. Data for HMM1 








Trans1 0.95, 0.05; 
0.10, 0.90 
estTR 0.905887481467734 0.0941 125185322665 
8.23612166133217e-07 0.9999991 76387834 
AvEstimatedTran 0.8819 0.1180; 
0.0861 0.9138 
Emis 0.25, 0.20, 0.10, 0.20, 0.25; 
0.05, 0.20, 0.50, 0.20, 0.05 
estEmis 0.290851170502618 0.265195169807112 1.86365327142984e-19 


0.340410481965135 0.103543177725135 
0.0780760494546633 0.159182284372720 0.325216993062660 0.307221304106831 
0.130303369003126 


AvEstimatedEmis 0.2429 0.1162 0.0148 0.3668 0.2590 ; 
0.1813 0.2315 0.3115 0.1961 0.0795 

Stationary Distribution of Trans1 xFixl = 0.6667 0.3333 

Stationary Distribution of xFixl = 0.4219 0.5781 


Av.estimated Trans] 





4.2. Estimation using maximum likelihood algorithm 

We also obtained the estimated transition probability and emission probability matrix by using 
the maximum likelihood (ML) estimate. In this process, MATLAB function uses the sequences and states to 
calculate the estimated matrixes. Table 4 shows the variation in data using maximum likelihood method for 
HMM1. If all the states are same then estimated transition matrix is single row and single column matrix 
and estimated emission matrix shows one row only. For example, for HMM1, if all the states are showing 
value “1” (channel is free) then Estimated transition matrix = 1. And estimated emission matrix is a row 
vector [0.24, 0.32, 0.12, 0.20, and 0.12]. HMM2 is more dynamic, where states are shifting fast. In HMM2 
case the maximum likelihood estimate is giving appropriate result. For example, in Table 4 estimateTRI1 is 
one of the estimated transition matrix and EstimateE3 is one of the estimated emission matrix. Table 5 shows 
the average of estimated transition matrices for both the HMM i.e. HMM1 and HMM2 using maximum 
likelihood method along with its stationary distribution. Figure 6 and Figure 7 shows the comparison of 
stationary distribution for HMM1 and HMM2 using both methods. 


Table 3. Data for HMM2 








Trans2 0.05, 0.95; 
0.90, 0.10 
estTR 2.18474853422124e-08 0.999999978152515 
0.860490176916748 0.139509823083253 
AvEstimatedTran 0.0418,0.9580; 
0.9461, 0.0537 
Emis 0.25, 0.20, 0.10, 0.20, 0.25; 
0.05, 0.20, 0.50, 0.20, 0.05 
estEmis 0.264496997053377 0.257120272324905 0.141836793599441 0.255723421934458 
0.0808225 150878192 
1.46499833377394e-32 0.0793448 105036851 0.687614729009560 
0.153723532194480 0.0793 169282922756 
AvEstimatedEmis 0.2542, 0.1752, 0.0639, 0.2557, 0.2508; 
0.0269 0.1726 0.6138 0.1598 0.0271 
Stationary Distribution of Trans2 xFix2= 0.4865 0.5135 
Stationary Distribution of Av. xFix2= 0.4969 0.5031 


estimated Trans2 





Table 4. Variation in data using maximum likelihood method for HMM1 








estimateTR 1 0.833333333333333 0.166666666666667 
0 1 
estimateTR2 0.909090909090909 0.0909090909090909 
1 0 
estimateTR3 0.954545454545455 0.0454545454545455 
0.500000000000000 0.500000000000000 
estimateTR4 ( all estimated states are “1”) 1 
EstimateE3 0.173913043478261 0.173913043478261 0.173913043478261 
0.260869565217391 0.217391304347826 
0 0 1 0 0 
EstimateE4( all estimated states are “1”) 0.24, 0.32, 0.12, 0.20, and 0.12 
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Table 5. Estimated average transition matrix and stationary distribution for HMM1 


and HMM2 using maximum likelihood method 
Hmm lavestiTRmaxlike 0.9014, 0.09846; 0.39422, 0.60575 
Hmm2avestiTRmaxlike 0.03333, 0.96664; 0.9218, 0.07818 
Stationary Distribution of HmmlavestiTRmaxlike xFixl= 0.8001 0.1999 
Stationary Distribution of Hmm2avestiTRmaxlike xFix2= 0.4881 0.5119 
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Figure 6. Comparison of stationary distribution for HMM1 using both methods 
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Figure 7. Comparison of stationary distribution for HMM2 using both methods 


5. CONCLUSION 

Here we have used Viterbi algorithm to find the hidden states from the given sequence. As HMM1 
changes slowly, state prediction percentage is higher as compare to HMM2. Still for hundred percentages 
result, we suggest “K out of N rule” so that predicted states would be correct. Spectrum occupancy detection 
in noisy channel is important task in cognitive network. Using Baum-Welch algorithm and maximum 
likelihood we have estimated the transition and emission probability matrices. If all the states are same then 
estimated transition matrix using maximum likelihood is single row and single column matrix and estimated 
emission matrix shows one row only. So for slowly changing channel this method gives confusing result. In 
terms of stationary distribution of the channel, maximum likelihood method gives good result than Baum- 
Welch algorithm, it is plotted in Figure 6 and Figure 7. Baum-Welch algorithm uses only sequences along 
with transition and emission matrices finds it comparatively difficult to estimates the transition and emission 
matrices. While ML method uses sequences and states to estimates both the matrices give comparatively 
good result in terms of stationary distribution if channel is not monotonous. HMM2 is more dynamic, where 
states are shifting fast, ML method gives comparatively good result as shown in Figure 7. In cognitive radio 
system, channel detection time is quite large (for IEEE 802.22 it is 2 sec.), also channel move time is there, 
so finding the averages is not a problem. Therefore in the estimation process, we suggest average of 
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estimated values for transition and emission matrices. Also state predicted by Viterbi algorithm for three 
times and taking the value which occurs two times improves the estimation of states. 
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